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Ihis paper provides descriptive information about: the 
implemenrarion of prescriptive criterion referenced testing and its 
impact on a compensatory reading laix>ratory program in the Houston 
public schools. Afrer describing the program and the rationale for 
xhe selection of a criterion referenced test, a study of the 
teachers' utilization of the rest and of pretesting and posttesting 
of program participants is ourlined. Eesults indicated that the 
reading specialists had gex^rally positive opinions about the 
criterion referenced test and the tesring program as implemented. A 
positive relarionship between degree of test utilization and 
individualizaricn of instruction vas suggested. High impleiientarion 
of rest resalrs produced Xiigher gains in pre/post comparisons. 
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The purpose of this presentation is to provide descriptive information 

aboat the impleiD.entation and ir^act of the use of prescriptive 

Criterion Referenced Testing on a coiE5>ensatory reading laboratory 
program in the Houston Public Schools. 

DESCRIPTION OF THE RROGPAM 

The Houston Independent School District Title I coirpensatory reading 
program uses a reading resource room approach. The rooms are divided 
into instructional centers, with selected materials and equipment 
focusing on the identified remedial needs* of students participating. 
These materials and equifxnent are not available to the regular 
classroom teacher. 

Students are identified as eligible to participate on the basis of 
test data using the criteria of approximately 1 standard deviation 
below the m^an grade equivalent for students at each grade level. 

These students are scheduled periodically throughout the day to 
receive intensive reading instruction in small groups or independently 
at centers located within the resource rooms. Each resource room is 
staffed with one reading specialist and one teachers aide who work 
with no more than 15 children per period. Children are grouj)ed and 
instructed according to specified needs as assessed by the tests. 

RATIONALE FOR SELECTION OF A CRITERION RETEREljCED TEST 

Evaluations of the 1974-1975 Title I eler^ntary reading program 
indicated a need for adding structure to the prograun. No real 
definition of the services incltided in the program was available. 
One recooaendation of the 1974-1975 Final Evaluation of the program 
was that the use of a prescriptive and diagnostic objective-based 
instrument might provide the needed structure. 

In the s^jrur.er of 1975 research personnel reviewed several Criterion 
Referenced Tests using the following criteria: 

1. Value to the teacher in planning for instruction. 
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2. Correspondence of test objectives vith the Houston 
Independent School District reading objectives*. 

3. Usability of the test for research piirposes. 

The test selected was presented to the Title I Supervisors by 
Research personnel and a coroany representative. Unanimous 
approval was obtained. 

DESCRIPTIOri C? THE TEST 

The Individualized Criterion Referenced Test for Reading (ICPJT), 
published by Educational Progress Corporation, was selected. 

The ICPT is based on sets of instructional objectives for reading. 
The continuum for reading contains 329 objectives. Administration 
of the ICRT requires the selection of 40 sequential objectives from 
the continuxin which are most appropriate to the functional reading 
level of the child. Decisions about which 40 objectives are appro- 
priate for a given child are based on teacher judgement and test 
data. 

For the 40 objectives tested, the scored tests indicate those 
objectives mastered, needing review, or not mastered. Administration 
of the ICRT as a pretest provides diagnostic information about 
skills needing instruction as well as indicating the reading skills 
acquired prior to entry in the program. Used as a posttest, the 
ICRT assesses the effectiveness of iirjstruction by indicating Tnastery 
of those objectives needing instruction which were identified on the 
pretest. 

DESCRIPTIOr: OF XNSERVICE TRAIIII^'G 

The local representative of Educational Develcpinent Corporation 
planned, with Houston Independent School District personnel, and 
presented three inservice programs for all Reading Specialists in 
the program. The topics included: 1) Test administration procedures, 
2) Interpretation of test results, and 3) Instructional techniques 
for use of the ICRT results. 

QPINIO!;S ABOUT ICRT AJ^D THE TESTING PROGRAM 

Evaluations of inservice training were conducted following each 
inservice sescicn. Specialists vere administered i/iservice evaluation 
questionnaires designed to assess opinions atout the quality of the 
inservice and the ICRT testing program as implemented within Houston 
Independent School District* 

Generally, questions airred at assessing inservice quality received 
high-mean ratings. A majority of the mean ratings to questions 
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assessing opinions about the ICRT testing program v/ere also high, 
however, more variable than those assessii^ the quality of the 
inservice- Higher mean ratings occurred to questions assessing 
opinions about clarity of instructions for test administration and 
use, the usefulness of test results for instruction and, test 
appropriateness for grades 3-6. Highest ratings were received by 
questions assessing the perceived usefulness of the material correla- 
tions and of the student printouts for planning. Approximately 80% 
of the specialists rated the usefulness of students* summaries for 
instruction as "much" or "very much". 

Lov^^est mean ratings occurred to questions assessing opinions about 
the time allotted for test administration and the appropriateness 
of the test for grades 1 s 2. Specialists felt the 8 day test 
administration period was unrealistic. Specialists also felt the 
test procedures for 1st S 2nd grade pupils was inappropriate, since 
ICRT readiness materials were not used by the district for these 
grade levels. 

In addition to assessing opinions through the inservice evaluation 
questionnaire, f iel i interviews were also^ conducted with a sample 
of 12 reading specialists. Responses to the interviews suggested 
a high degree of satisfaction with the test as well as high ratings 
of test accuracy. The specialists reported a high degree of satis- 
faction with the test as indicated by a mean rating of 5.6 on a 
7 point rating scale. All specialists agreed that Houston Indepen- 
dent School District should continue the use of the ICRT for the 
reading program, with several specialists indicating that the test 
was superior to- those previously provided . in the program. Specialists 
were also asked to rate on a 7 point scale the overall accuracy of 
the test. The responses ranged from 4 to 7 with a mean rating of 5.6. 
While most specialists felt the test was accurate, 10 of the 12 also 
indicated that students had skills needs in addition to those diagnosed 
on the test. 

TEST UTILIZATIOr; 

An examination of test utilization was initiated through the use of 
field interviews and observations. All specialists interviewed 
reported: 1) using the test results for either grouping, diagnosis, 
prescription, or individualization, 2) using the correlated materials 
from the printouts and 3) sharing the test information with either 
the regular classroom teachers, the students, or the principal. 
Utilization of the correlated materials was reported by all specialists, 
however, all indicated that materials other than those correlated 
were also necessary-. Estimates of the percentage of materials used 
m instruction which were not correlated on the printout ranged 
between 10% and 90%. 
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Regarding the adequacy of the materials correlated, 9 of the 12 
specialists indicated the correlated material to be adequate" in 
reinforcing the skills ns*»ds assessed on the test. Specialists 
listed inappropriate levels of skill reinforcement and the introduc- 
tion of several skills in addition to those correlated, as inadequacies 
in the material correlations. 

Responses to questions in the interview indicated a high degree of 
reported test utilization. Field observation did not substantiate 
the high degree of test utilization reported by teachers. Three 
types of observations were employed in assessing tests utilization- 
Accessibility . and availability of the printouts to the teacher was 
noted. Printouts were examined to determine any evidence of use 
such as writing on the printouts or excessive wear on the pages. 
The instructional activities offer-d by each specialist to a sample 
of 3 students were observed and recorded. The specific reading 
skills addressed by these activities were compared to the skills 
indicated on the printouts. The results of the observations revealed 
that the printouts were easily -accessible to 11 of 12 specialists. 
The printouts of 9 of 12 specialists had some evidence of use. In 
addition, there was correspondence between the skills addressed in 
instructional activity and the skills needs identified on the ICRT 
for only 7 cf the 12 specialists. These observations suggest that 
the actual incidence of test utilization is lower than that reported 
by the specialists. 

ItlDIVIDUALIZATIOrJ OF iNSTRaCTIGN 

In addition to observations relating to test utilization, a modified 
version of an observation and analysis instrument developed by 
Ben M. Harris and Kenneth E. Mclntyre from U.T. was utilized in 
obtaining ratings of individualization of instruction for the 12 
sample teachers. One instructional period was observed for each 
teacher and ratings made on the 19 item instrument. The items 
selected for use represented four dimensions important to individualized 
instruction. These include: intra- class grouping, variety of 
materials, pupil autonomy, and differentiated assignments. Ratings 
on each item were converted to nanerical values and su^Tmed to derive 
a total score for individuali/:ation. A score of 97, representing 
high individualization, was the highest possible score and 19 the 
lowest possible score. The total scores for individualization of 
instruction ranged from 23 to 74 with a mean of 43.4. ' VThile the 
size of the sarple did rot penrdt statistical analysis of the relation- 
ship between test utilization and individualization, it is interesting 
to note that the mean total score for individualization for those 
teachers with clear evidence of test utilization was 54 while the 
mean total score for those specialists with little evidence of use 
was 28. 
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RESULTS OF PP£~POSTTE STING 

A sairole of 72 second grade and 87 fifth grade progra:.: participants 
receiving remedial instruction frcxn the 12 teachers observed were 
pre and posttested on the ICRT. Each child was pretested over a 
dxrferent set of 40 objectives appropriate to his functional reading 
level, and posttested over those same objectives. Three scores 
were obtained: 1) the number of objectives needing instruction 
on the pretest which were mastered on the posttest, 2) the number 
of objectives needing review on the pretest which were mastered on 
the posttest and 3) a total score. The test provides three types 
of information on objective mastery, including comp! '-te mastery, 
needing review, and needing instruction. For resear./h purposes we 
have derived a total score which increases as the child moves toward 
mastery of objectives and decreases as he moves away from mastery 
toward needing instruction. The results of sample pre and posttesting 
for each of the scores by grade are presented in Table I below. 

TABLE I 

ME All JAINS SCORTS ON HIE ICRT BY GRADE 



Grade 



Needing Instruction 
1) Pre-Mastery Post 



Needing Review Total 
2) Pre-Mastery Post | 3) 



4.60 



3.16 



6.89 



5.65 



12.82 



8.79 



As can be seen in Table I, the mean gains of second graders were 
greater than those of fifth graders. Based on the results of an 
analysis of variance with grade as a factor, the mean differences 
apparent in the Table were found to be statistically significant. 

Observations of test utilization were combined with ratings of 
individualization of instruction in subdividing the 12 teachers 
into two groups of six. Teachers with obvious evidence of use and 
high scores on individualization were collapsed into a group and 
labeled "high implementers" . Specialists with less evidence of 
test use and relatively lower scores on individualization of 
instruction were labeled "low implementers" . Test scores of students 
receiving instruction from teachers in each of these categories were 
compared in an analysis of variance. The mean gain scores for the 
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"high" and "low impleinenters" are presented in Table II. All mean 
differences were foxmd to be statistically significant. ' 

TABLE II 

KEAIJ GAINS FOR "HIGH" AND "LOW IMPLEI-ENTERS" 



Implementation 


Needs Instruction on 


Needs Review on 


Total 




Pre-Mastery on Post 


Pre-Mastery on Post 




High 


4.90 


7.36 


15.37 


Low 


2.30 


5.12 


6.14 



As can be seen in Table II, "high implementers" had higher mean gain 
scores than "low implementers". It appears from these data, that 
significantly greater gains are achieved from specialists who utilize 
the ICRT results and individualize instruction. 

CONCLUSIONS 

1. Houston Independent School District specialists have generally 
positive opinions about the ICRT for reading and the testing 
program as implemented. 

2. ^A positive relationship between test utilization and individual-: 

ization of instruction was suggested. - Additional data on a 
larger sample is necessary before a determination can be made. 

3. High implementation of test results produced higher gains in 
objective mastery. 



